The present invention relates to a moving picture editing apparatus for use in production of such a moving picture program as television broadcasting program or video program.
The performance of a facility for television broadcasting programs or video programs has conventionally been advanced rapidly with the help of computer. In these years, in particular, remarkable advancement is directed not to a video editing method of a video tape recorder based on repeating fast-forward and fast-reverse depending on a tape counter, but to a moving picture editing apparatus for storing source material video pictures in an inexpensive hard disk which is easy to handle and save for editing of source material video pictures, that is, to a so-called nonlinear editing apparatus.
FIG. 2 shows a block diagram of a nonlinear moving picture editing apparatus.
The illustrated apparatus includes a scene change point detector 201, a magnetic memory device 202, a monitor 203, a frame buffer 204, a main memory 205, an input device 206, a video tape recorder (VTR) 207, a video interface 208, a picture compressor 209, an audio interface 210, a central processing unit (CPU) 211, a bus 212 and a picture size reducer 214.
The scene change point detector 201, magnetic memory device 202, monitor 203, frame buffer 204, main memory 205, input device 206, VTR 207, audio interface 210, picture compressor 209 and picture size reducer 214 are connected to the CPU 211 through the bus 212. The VTR 207 is connected to the video interface 208 and audio interface 210.
In FIG. 2, the CPU 211 performs data transfer with the respective constituent elements connected thereto via the bus 212, the constituent elements being controlled by an access signal received from the CPU 211. An instruction to the CPU 211 is carried out by an operator with use of the input device 206 such as a mouse or keyboard. The VTR 207 reproduces a series of moving picture information containing plurality of scenes, sends moving picture in the reproduced information to the video interface 208, and also sends an audio signal in the reproduced information to the audio interface 210. The audio interface 210 converts the received audio signal to a digital signal and sends the digital signal to the magnetic memory device 202. The magnetic memory device 202 stores therein the received audio information. The video interface 208 converts the received moving picture information to information having a format processable by the moving picture editing apparatus, and sends the converted information to the scene change point detector 201, magnetic memory device 202, main memory 205, picture compressor 209 and picture size reducer 214. The picture compressor 209 compresses digital data of the reproduced moving pictures, e.g., based on a moving picture experts group (MPEG) or motion joint photographic experts group (motion JPEG) system. The magnetic memory device 202 stores therein the received video information. The scene change point detector 201 analyzes the received moving pictures on a frame basis and detects a scene change point and a cut change point on the basis of a degree of change in a color distribution between frames. The picture size reducer 214 generates a reduced moving picture (M-icon) by thinning out data of each frame into video or moving picture data of 80xc3x9760 pixels on the basis of an analysis result of the scene change point detector 201. The generated M-icon is sent to the magnetic memory device 202 and stored therein. Moving picture analysis information (including frame numbers for scene change points or cut change points) indicative of a relationship between the M-icon and the detection result of the scene change point detector 201 is also sent to the magnetic memory 202 and saved therein. The monitor 203 displays the moving pictures stored in the magnetic memory device 202 on the basis of analysis information thereof to offer a graphical user interface (GUI) to the operator. The frame buffer 204 stores therein pictures to be displayed on the monitor 203. The main memory 205 stores therein various sorts of control programs for the CPU 211.
In this connection, the magnetic memory device 202 may be replaced by another recording medium such as a magneto-optical memory or a remote file connected through a network. Further, the mouse used as the input device 206 may be replaced by a track ball or another pointing device. A communication device 215 may be a Ether Net, an interface, asynchronous transfer mode (ATM) interface, a modem, a facsimile machine, a telephone or another communication terminal which can perform information transfer between an external communication network and the nonlinear editing apparatus.
The operator sequentially edits the reduced moving picture (M-icon) on which the scene analysis information obtained at the scene change point detector 201 is reflected, and constructs them into a tree structure.
FIG. 3 shows an example of the editing display screen displayed on the monitor 203. In the drawing, reference numeral 301 denotes a hierarchical structure (a tree structure) area, numeral 302 denotes an M-icon, 303 denotes a schematic enlargement of the M-icon, 304 denotes a source material area, and 305 denotes a group of various function buttons. Editing is carried out by constructing M-icons into such a hierarchical structure as shown in the hierarchical structure area 301 of an editing screen 300. In the tree (in the hierarchical structure area 301), M-icons (for example M-icon 302) of a lowest layer are cuts which are composed a plurality of frame images, M-icons of a upper layer of said lowest layer are scenes which are composed a plurality of cuts, and a M-icon of a uppermost layer are moving pictures which are composed a plurality of scenes. Namely, in the tree having a plurality of layers, an M-icon is in upper layer represents a combination of M-icons in a lower layer belonging to the upper layer, which corresponds to a chapter or section in a story. An M-icon in the uppermost layer represents the story. The source material area 304 is an area in which the M-icons 302 prepared by the picture size reducer 214 based on an analysis result of the scene change point detector 201 are arranged. The operator selects ones of the icons to be edited and constructs them into a hierarchical structure in the hierarchical structure area 301. The various function buttons 305 are operating buttons for various sorts of editing operations on the editing screen 300.
Such an apparatus as mentioned above can lighten operator""s burden.
More in detail, since editing can be realized on a scene or cut (or clip) basis, story making can be facilitated. Further, since division of the input moving pictures into cut units can be automatically carried out, operator""s inconvenience of manually doing so can be eliminated. In addition, since the moving pictures can be stored and managed in the form of a hierarchical structure, retrieval of a necessary scene or cut (or clip) can be facilitated.
Explanation will next be made by referring to FIGS. 4 and 5 as to a specific example (information managing/displaying method) of a method for managing related video information (linked video information) in the prior art nonlinear editing apparatus. FIGS. 4 and 5 show related video linked information such as source material names, types and lengths of source material media, and comments. Reference numeral 401 denotes a column for indication of different sorts of source material icons, reference symbol 401-1 and 401-2 denote source icons, numeral 402 denotes a column for indication of names of the source materials, 403 denotes a column for indication of a distinction between video and audio, 404 denotes a column for indication of time durations of the materials, 501 denotes a column for indication of typical source material icons, 502 denotes a column for indication of title names, types and durations of the typical source icons, 503 denotes a column for indication of comments of the materials for their editing, and 504 denotes a column for indication of labels of the source materials.
In FIG. 4, the source icons column 401 shows the source icon 401-1 and source icon 401-2 indicative of source material types. For example, the source icon 401-1 indicates that the source material is a broadcasting program, while the source icon 401-2 indicates that the source material is a scene. Other source icons include an icon indicative of one cut or clip. The source material name column 402 shows names of the source materials given in the source icons column 401 at the left side of the column 402. The video/audio distinction column 403 indicates xe2x80x9cA1-2xe2x80x9d when the source material is of an audio type, and indicates xe2x80x9cV1xe2x80x9d when the source material is of a video type. Further, the material duration column 404 given at the right side of the column 403 indicates time lengths or durations of the source materials.
In FIG. 5, the typical material icon column 501 shows therein an audio source icon 501-1, a still image source icon 501-2, a moving image source icon 501-3 accompanied by audio information, a folder icon 501-4, and a moving image source icon 501-5 not accompanied by audio information. The name/type/duration column 502 given at the right side of the column 501 indicates names, types, time lengths, etc. of the source materials given in the typical material icon column 501. For example, the column 502 at the right side of the audio source icon 501-1 shows xe2x80x9cnamexe2x80x9d, xe2x80x9cmoving imagexe2x80x9d and xe2x80x9ctime durationxe2x80x9d of the source material in question, while the column 502 at the right side of the still image source icon 501-2 shows xe2x80x9cnamexe2x80x9d, xe2x80x9cstill imagexe2x80x9d and xe2x80x9ctime durationxe2x80x9d of the source material in question. The column 502 a the right side of the moving image source icon 501-3 accompanied by audio information indicates or at the right side of the moving image source icon 501-5 not accompanied by audio information shows xe2x80x9cnamexe2x80x9d, xe2x80x9cmoving imagexe2x80x9d and xe2x80x9ctime durationxe2x80x9d of the associated source materials. The column 502 at the right side of the folder icon 501-4 shows xe2x80x9cfolder namexe2x80x9d, xe2x80x9cfolderxe2x80x9d and xe2x80x9cnumber of source materials in the folderxe2x80x9d. Further, the comment column 503 describes editing states of the associated source materials or simple information necessary for the associated source materials, and the source material label column 504 describes label names of the source materials. Examples of a nonlinear editing apparatus are disclosed in xe2x80x9cMedia Suite Pro 1.0 User""s Guidexe2x80x9d by Avid Technology, Inc., 1994, pages 92, 93, 123 and 125 and in xe2x80x9cAdobe Premiere 4.2 User Guidexe2x80x9d by Adobe Systems Incorporated, 1997, pages 50 and 55.
In a prior art moving picture editing apparatus called the nonlinear editing apparatus, information accompanied by video information is limited only to the time length of a video source material or producer""s comment and can be controlled only so as to give uniform information to respective video source materials. For this reason, the prior art apparatus cannot cope with such various types of video-accompanied information a plurality of layers that are different degrees of abstraction.
In the course of video production, a large amount of individual information related to video production is actually generated in various formats of information, including, for example, such input related information as input date, input location and input intention, such right related information as the right of likeness or portrait, the copyright on music, usable range and use expiration date, such video additive information as film script and caption, and such edit associated information as edit history and time code.
With respect to such video accompanied information, tape based editing (the linear editing apparatus) has been so far conducted in a broadcasting station in such a manner that a video producer keeps these information in different places in the form of a memo note or pad about data collection, editing and broadcasting memo or in the form of a word processor floppy disk. Further, video information per se has been stored in video tape. In this way, information has been stored in different media depending on the type of the information. Therefore information management has had to be separately carried out and thus relations and correlations between the information have had been ambiguous with time passage, leading eventually to scatter and loss of the information. The same situation has been occurred in the conventional linear editing system.
It is therefore an object of the present invention to provide an apparatus for editing moving picture having an related information called an integrated information management system which can relate image units of a moving picture to additive information associated with the image units and having various types of information.
In accordance with an aspect of the present invention, the above object is attained by providing an apparatus for editing moving picture having related information thereof. This apparatus comprises input means for inputting said moving picture and said related information, said moving picture comprising a plurality of scenes, each scene comprising a plurality of cuts and each cut comprising a plurality of frame images; detecting means for detecting change points of said moving picture and dividing said moving picture into said scenes and cuts; selecting means for extracting a still image representing each of said scenes and cuts; a memory for storing said related information and the representative still images of said scenes and cuts; means for designating said representative still images of said scenes and cuts to edit said moving picture; means for determining a hierarchical structure, said hierarchical structure comprising a plurality of layers in accordance with said scenes and cuts; means for displaying said hierarchical structure and said related information; and a controller for controlling said memory and said display means so that said related information in relation to said representative still images to be displayed is displayed with said representative still images.
In accordance with another aspect of the present invention, the above object is attained by providing a method for integratedly managing a moving picture and additive information under control of a computer, which comprises the steps of displaying selected typical ones of frame images of the moving picture on a display device in the form of a hierarchical structure, and displaying additive information associated with the moving picture on the display device in the form of representation of degrees of abstraction different according to the layers of the hierarchical structure, as associated with the displayed typical frame images.
In accordance with a further aspect of the present invention, there is provided a computer program product which includes a recording medium in which a procedure in the above integrated information management method is recorded in the form of a format readable by the computer.
In the present invention, the typical frame images of the moving picture are related to the respective additive information associated with the images, the additive information is displayed on the screen of the display device as associated with the frame images, and also displayed in the form of representation of different degrees of abstraction different according to the layers of the frame images associated with the additive information in the hierarchical structure of the moving picture. Therefore, the moving picture and additive information can be all managed one-dimensionally on the screen, and thus can support the editing operation of the moving picture so as to match editor""s thinking pattern.